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OPTIMAL STOPPING AND FREE BOUNDARY 
CHARACTERIZATIONS FOR SOME BROWNIAN 
CONTROL PROBLEMS 1 

By Amarjit Budhiraja and Kevin Ross 

University of North Carolina at Chapel Hill and Stanford University 

A singular stochastic control problem with state constraints in 
two-dimensions is studied. We show that the value function is C 1 
and its directional derivatives are the value functions of certain opti- 
mal stopping problems. Guided by the optimal stopping problem, we 
then introduce the associated no-action region and the free boundary 
and show that, under appropriate conditions, an optimally controlled 
process is a Brownian motion in the no-action region with reflection 
at the free boundary. This proves a conjecture of Martins, Shreve and 
Soner [SI AM J. Control Optim. 34 (1996) 2133-2171] on the form of 
an optimal control for this class of singular control problems. An im- 
portant issue in our analysis is that the running cost is Lipschitz but 
not C 1 . This lack of smoothness is one of the key obstacles in estab- 
lishing regularity of the free boundary and of the value function. We 
show that the free boundary is Lipschitz and that the value function 
is C 2 in the interior of the no-action region. We then use a verifica- 
tion argument applied to a suitable C 2 approximation of the value 
function to establish optimality of the conjectured control. 

1. Introduction. We consider a singular stochastic control problem with 
state constraints in two-dimensions. Roughly speaking, by singular control 
one means that the control terms in the dynamics of the state process need 
not be absolutely continuous with respect to the Lebesgue measure, and 
are only required to have paths of bounded variation. State constraints, a 
key feature of our problem, refer to the requirement that the controlled 
diffusion process takes values in some proper subset S of M?. More precisely, 
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in our setting § = and the state process is described by the equation 
X = x + B + Y, where x 6 §, B is a two dimensional Brownian motion 
with drift 9 and nondegenerate covariance matrix £ and the control Y is a 
nondecreasing, right continuous with left limits (RCLL), adapted process. Y 
is said to be an admissible control if X(t) £ S for all t > 0. Associated with 
each initial state and control policy is an infinite horizon discounted cost 



Jo 

where 7 € (0, 00) is the discount factor and £:§— > R + is a convex function 
of the following form: For z = {z\, Z2)' € S, 



where a, (3 £ M 2 , and c £ (0, 00). The value function V(x) is the infimum of 
J(x, Y) over all admissible controls. 

Such a control problem, and its connections with queuing networks in 
heavy traffic, has been studied by many authors [5, 8, 12, 18, 20, 21]. In a 
general multidimensional setting and with a much more general cost func- 
tion, such control problems were studied in [1] and [6]. In [1] the value 
function was characterized as the unique viscosity solution of an appro- 
priate Hamilton-Jacobi-Bellman (HJB) equation, whereas [6] established 
the existence of an optimal control by general compactness arguments. Our 
main contribution in this work is to provide an explicit representation for 
an optimal control under appropriate conditions on I. 

Explicitly solvable singular control problems are quite rare. In the few 
examples where explicit solutions are available, one finds that an optimal 
control takes the following form. There is an open set O in the state space 
such that starting from within O no control is applied until the state trajec- 
tory reaches the boundary dO, at which point a minimal amount of push is 
applied along an appropriate control direction to constrain the state process 
within O. Furthermore, if the initial condition is outside O, an instantaneous 
jump occurs at time that brings the process to dO and, subsequently, con- 
trol is applied as described above, constraining the process within O. In 
other words, an optimally controlled process is a reflected diffusion on O 
with an appropriate (possibly oblique) reflection field. In terms of the asso- 
ciated HJB equation, in O the value function satisfies a linear elliptic PDE 
and in O c a nonlinear first order PDE is satisfied; the boundary dO, sepa- 
rating these two regions, is referred to as the free boundary for the system of 
PDEs. Such characterizations for optimal controls of singular control prob- 
lems in terms of a diffusion reflected at the free boundary are some of the 
most useful and elegant results in the field. For one dimensional settings 
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Z2 > CZ\ 
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there have been several works (see, e.g., [3, 11, 14]) that have used the so- 
called principle of smooth fit to establish the C 2 property of value functions 
of certain singular control problems and then characterize the free bound- 
ary and an optimally controlled process. In more than one dimension the 
only such results are due to Shreve and Soner [25, 26]. As one may expect, 
such results are intimately tied to regularity (i.e., smoothness) properties 
of the free boundary, which in turn hinge on similar properties of the value 
function of the control problem. For example, in [25] the authors consider a 
two dimensional singular control problem in K 2 (in particular, there are no 
state constraints) with dynamics governed by the equation X = x + B + Y, 
where B is a two dimensional Brownian motion and Y is a RCLL control 
with paths of bounded variation. The goal is the minimization of the cost 
E Jj ^ e (£(Xt) dt + d\Y\t). Under strict convexity of t and suitable growth 
conditions on its first two derivatives, the authors first establish, using ideas 
of Evans [9] and Ishii-Koike [13], that the value function V is a C > so- 
lution of the PDE: max{(-A + 1)/ - £, \Df\ 2 - 1} = 0. In a construction 
that essentially uses the two dimensional nature of the problem the au- 
thors are then able to use the gradient flow of V to upgrade the regularity 
of V to C 2 . This regularity, in conjunction with results of Caffarelli [7] and 
Kinderlehrer and Nirenberg [17], is then used to show that the free boundary 
dO = {x: \DV\ = 1} is C 2 ' a for any a £ (0,1) and an optimally controlled 
process is a reflected diffusion in the region {|.DV| < 1} with the oblique 
direction of reflection —DV{x) at x G dO. The existence and uniqueness of 
such a reflected diffusion follows from the established smoothness of dO and 
the reflection field —DV (see [19]). 

Two main differences from [25, 26] in the current setting are the state 
constraint requirement on admissible controls and the lack of regularity of 
the running cost I. Note that the cost in (2) is neither strictly convex nor C 2 
(in fact, it is not even C 1 ). These difficulties make C 2 regularity of the value 
function an unrealistic goal. Nevertheless, exploiting the convexity of £, we 
show in Section 3 that the value function is C 1 in 8° and the gradient of the 
value function extends continuously to all of §. Our proof of C 1 regularity 
is probabilistic and a key ingredient to the proof is the availability of an 
optimal control as established in [6] (see proofs of Lemmas 3.3 and 3.4). We 
next turn to the study of the free boundary problem and a representation for 
an optimally controlled state process. In the case where a > and (5 > 0, one 
finds that an optimally controlled process is a Brownian motion, reflected 
normally on the positive quadrant (see [5, 20]) and, thus, the free boundary 
is {x : x\ = or X2 = 0}. In this case the C 2 property of the value function in 
§° follows from classical elliptic regularity results, as was noted in [20]. Thus, 
the interesting cases correspond to the setting where at least one coordinate 
of the parameters a or (3 is negative. (Note that the assumptions on I imply 
additional restrictions on the parameters a and f3.) 
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In Sections 4 and 5 of this paper we will focus on the case a ^ 0, /3 > 0. 
[By a 0, where a = (01,02), we mean that Oi < for at least one i = 1,2. 
Similarly, by (3 > we mean that /?j > 0, i = 1, 2, where j3 = (/3i , /?2)-] in the 
queueing network setting this parameter regime corresponds to Case IIB 
of [20]. In the notation of that paper, a\ = C2H2 — £3^2, «2 = C3/X3, (3\ = C\^i, 
02 = A i 3(c2A f 2 — c\Hi) / [12, where ^1,^2,^3 correspond to the service rates and 
ci, C2, C3 to the holding costs of the queueing network model. The parameter 
regime a > 0, (3 ^ (Case IIC of [20]) can be treated in a symmetric manner. 
Finally, the case a ^ 0,f3 ^ (Case IID of [20]) appears to be a significantly 
harder problem and is beyond the scope of the current study. 

In Section 4 we show that the xi-directional derivative of the value func- 
tion is the value function of a closely related optimal stopping problem. Con- 
nections between singular control problems and optimal stopping/obstacle 
problems (see [24]) were first observed by Bather and Chernoff [2] and, sub- 
sequently, such correspondence results have been studied by several authors 
[4, 15, 16, 25, 26] in one-dimensional and certain multi-dimensional models. 
The paper [16] is the only other paper that studies such connections in the 
presence of state constraints. The key differences between [16] and our set- 
ting are that in [16] the cost function is assumed to be C 1 and the main 
correspondence result is established under the assumption that the control 
problem admits an optimal solution. 

The study of the optimal stopping problem suggests that the "no action 
region" for an optimal control policy should be given as 

G = {x e S:xi > ^(x 2 ),X2 > 0}, 

where 

= sup{z! > : V 1 V(z 1 ,z 2 ) = 0}, z 2 > 0, 

with ViV denoting the partial derivative of V in the direction e\. We show 
that ^ : R + 1— > M + is a nondecreasing Lipschitz continuous function with 
Lipschitz norm bounded by c~ 1 , ^(0) = 0, and that ^(z2) — ► 00 as Z2 — > 
00. A natural conjecture for an optimally controlled process is a Brownian 
motion in G, reflected at dG, where the direction of reflection is e2 on c^G = 
{x G S : x ■ e2 = 0}, whereas on d\G = {x G S : ^(2:2) = x±} the direction of 
reflection is e\ (see Theorem 5.2). A similar conjecture, without giving a 
precise description of was first formulated in [20]. A major obstacle in 
showing that the conjectured controlled process is optimally controlled is 
the lack of sufficient smoothness of the free boundary (^ is only Lipschitz) 
and the value function. Typical proofs of such a result (see [25, 26]) follow 
through an application of Ito's formula using the fact that the value function 
is a classical solution of the associated Hamilton-Jacobi-Bellman (HJB) 
equation. In view of unavailability of enough regularity, we proceed with a 
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viscosity solution approach. It was established in [1] (see Theorem 5.4 of the 
current work) that the value function is a constrained viscosity solution of 
the nonlinear PDE (42). From this and standard elliptic regularity results, 
we obtain that the value function is C 2 and is a classical solution of a 
linear elliptic PDE (43) on G°. For the candidate optimal control policy, 
denoted as Y*, when initial point x € G, the control term Y* increases only 
when the process is at the boundary diG. Also, one finds that for x £ diG, 
ViV(x) = 0. Thus, formally applying Ito's formula to V with the candidate 
optimally controlled process X*, one obtains that 



The desired optimality of X* then follows on sending t — ► oo. The main 
difficulty in the proof is that due to the lack of sufficient regularity of the 
value function on dG, we cannot apply Ito's formula directly to V. In order 
to make the above argument rigorous, we consider an approximation V s of 
V that is C 2 in an open set containing G, apply Ito's formula to V s , and 
finally send e — > 0. 

In [20] the authors provide a conjecture for an optimally controlled process 
in the case a^0,/3^0 as well. In this case, the boundary of the no-action 
region would be determined by two functions and ^ with properties 
analogous to those of ^ described in the previous paragraph. An optimal 
control would apply reflection along the free boundary and an optimally 
controlled process would be described by a set of coupled equations similar 
to (40). We refer the reader to [20] for the precise form of this conjecture in 
the queueing network setting. Analysis of this parameter regime will be the 
subject of future research. 

The paper is organized as follows. We present the singular control prob- 
lem and summarize some key properties of its value function in Section 2. 
Section 3 is devoted to the proof of Theorem 3.1 which establishes the C 1 
property of the value function. Sections 4 and 5 study the case a ^ 0,/3 > 0. 
In Section 4 we introduce an optimal stopping problem and prove in Theo- 
rem 4.1 that the xi-directional derivative of the value function of the singular 
control problem equals the value function of the optimal stopping problem. 
We introduce in Section 5 the free boundary associated with the singular 
control problem and the conjectured form of an optimal control policy. The 
main result is Theorem 5.2, which establishes that an optimally controlled 
process is a Brownian motion in the no-action region G° with reflection at 
the free boundary dG. 

We will use the following notation. The set of nonnegative real numbers 
is denoted as R+. For i£l 2 , |x| denotes the Euclidean norm. The standard 
orthonormal basis in R 2 will be written as {ei,e2}. All vectors are column 
vectors and vector inequalities are to be interpreted componentwise. Given 
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a metric space E, a function / : [0, oo) — > E is RCLL if it is right-continuous 
on [0, oo) and has left limits on (0, oo). A (stochastic) process is RCLL if its 
sample paths are RCLL a.s. If O is an open subset of M 2 and / : O i— >• K is 
differentiable, then Vj/ denotes the partial derivative of / in the direction 
ej, i = 1,2. The class of twice continuously differentiable functions on O will 
be denoted as C 2 {0). 

2. Setting. Let B be a two dimensional {J-t }-Brownian motion with drift 
9 and nondegenerate covariance matrix £ given on some filtered probabil- 
ity space (fi,jr,{^},P). We will denote (ft, JT, {j^}, P, £) by $ and call it 
a system. The state space § of the controlled process X, introduced be- 
low, is the positive quadrant Given x € §> and an RCLL, {jF t }-adapted, 
nonnegative, nondecreasing process Y, define 

(3) X(t) = x + B(t) + Y(t), t>0. 

We say that such a process Y is an admissible control for the initial condition 
x if X(t) £ § for all t > 0, a.s. The class of all admissible controls (for 
the system <£) will be denoted by A(x, <&). We consider an infinite horizon 
discounted cost J(x,Y) defined as in (1), where > M + is a continuous, 
convex function defined as in (2). Note that the nonnegativity and convexity 
of I imposes additional conditions on the values of a, (3, c which are not made 
explicit here. 

The value function of the control problem for initial condition x € S is 
defined as 

(4) V ( x ) = ™{ JfrY), 

where the outside infimum is taken over all probability systems <]?. 

Next we record some useful properties of the value function. Let $ be 
an arbitrary system and let Tf be the P-completion of a{B(s) : < s < t}, 
the filtration generated by B. We will write the system (£l,J-, {jT t B },P, B) 
as < 1 )B . The following result was established in [1]. 

Proposition 2.1 (cf. Theorem 2.1 of [1]). For all x £ S, V(x) = 
ini YeA(x,**) J(x,Y). 

The proof of the following lemma is contained in Lemma 4.5 of [1]. 

Lemma 2.2. V is finite and Lipschitz continuous on S. 

The following elementary lemma establishes the convexity of V . 



Lemma 2.3. V is a convex function on S. 
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PROOF. Fix x® eS, i = 1,2, and let A G [0, 1]. Set x = Xx^ + (1 - 
A)x( 2 ). It suffices to show that V(x) < AF(xW) + (1 - A)T/(x( 2 )). Let e > 
be arbitrary. Fix a system $ and let $ B be as introduced above Proposition 
2.1. Then one can find yW G „40 (i) , $ B ), j = 1,2, such that J(xW,yW) < 
V(xW) +e. Clearly, Y" = AY^ + (1 - A)y( 2 > G .A(x,$ B ). Furthermore, the 
convexity of I yields 

V(x) < J(x, Y) < AJ(x«, Y«) + (1 - A)j(x( 2 ),y( 2 )) 

< AV(x (1) ) + (l-A)y(x (2) ) + e. 

Since e > is arbitrary, the result follows. □ 

The following result on the existence of an optimal control was established 
in [6]. The result will be used in the proofs in Section 3. 

Proposition 2.4. Let x e S. Then there exists a system 3? and Y* G 
A(x, $) such that V(x) = J(x, Y*). 

PROOF. In the notation of [6], let W = U = G be the two-dimensional 
identity matrix, and h = 0. Then equation (5) of [6] is satisfied with ag = 1 
and Condition 2.2 of [6] is satisfied with cq = 1. Thus, the result is an im- 
mediate consequence of Theorem 2.3 of [6]. □ 

3. C 1 - property of the value function. The main result of this section 
is the following C 1 property of the value function. 

Theorem 3.1. For each x G §°, ViV(x), i = 1,2 exist. The functions 
x i— > ViV(x) are continuous on S° and can be continuously extended to all 
of S. 

In proving the above theorem, we will only consider ViV; the proof for 
the existence and continuity of V2V is carried out in a symmetric fashion. 
For x G S, define 

V + V(x) = lim— ^ — . 

sio 5 

Similarly, for x G § such that x ■ e\ > 0, define 

r, „/ n • ,. V(x) - V(x - Sex) 
V V(x) =lim— ^ ^ —. 

610 5 

Existence of the above limits is a consequence of convexity of V (see Theo- 
rem 24.1 of [23]). The following lemma is also an immediate consequence of 
convexity of V. For a proof, see Theorem 24.1 of [23]. 
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Lemma 3.2. Let xeS with x • e x > 0. Then VV{x) < V + V(x). 

Fix x G § with x • e\ > 0. In view of Lemma 3.2, to establish the existence 
of V±V(x), it now suffices to show that V + V(x) < V~V{x). This inequality 
will be established by considering the following auxiliary control problem. 
From Proposition 2.4 one can find a system $ and Y* G A(x, <3?) such that 
V(x) = J(x, Y*). Denote the corresponding state process by X* . For the rest 
of this section we will fix such a (<]?, Y*,X*). Define the R 2 valued stochastic 
process Z = (Z±, Zq)' as 

Z 1 (t) = x 1 + B 1 (t); 

(5) 

Z-2.it) = X*(t) =x 2 + B 2 {t) + Y*(t), t > 0. 
Define S = inf {t > : Z(t) ■ e± < 0} and for a given {^}-stopping time a set 

raAS 

(6) J(x,a)=E e-^£(Z(t))dt, 

Jo 

where, for 2£S, 

(?) i(*) = {T eu Z2 - CZ1 ' 

Note that ^ is the left derivative of ^ (which exists due to the convexity of 
t) in the ei-direction, that is, 

w <5|0 5 

Also, convexity of £ gives that ^ is nondecreasing in the zi variable; in 
particular, a ■ e\ < (3 ■ e±. 
Define 

(8) u(x) = sup J(x,a), 

<T6S($) 

where 5($) is the set of all {JT f }-stopping times. 

The following lemma is the first key step in the proof of Theorem 3.1. 

Lemma 3.3. For x G § with x ■ e\ > 0, \7~V(x) > u(x). 

Proof. Let 5q > be such that x$ = x — 5e\ G § for all 5 < So. Fix 
5 < 8 . Define 5a = inf{i > 0:Z(t) ■ e x < 5} and let for t > 0, a G 5($), 
y 5 (t) = y*(t) + <5eil {t > CTA5i} . Set 

= x 5 + + Y 5 (t) = x + B{t) + Y*{t) - <5 ei l {0 < t<(TA5a} . 
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V(x s ) < J{x 5 ,Y s ) = E / e^H{X 5 (t)) dt + E e-^£{X 5 {t)) dt. 

JO JaASg 

Since £ is convex, we have (see Corollary 24.2.1 of [23]) that for z£§ such 
that z — Sex € S, 

(9) £{z - 5ei) - £{z) = -5 f £{z - uSe x )du. 

Hence, 



o 



(io) ~ 5 J \ aSs J \ 

> E [ " e~ 7i / £(Z(t) - Suex ) du dt, 
Jo Jo 

where the last inequality uses the fact that £(z±,Z2) is nondecreasing in z\. 

From the sample path continuity of Z • ei, we see that Sg T S as 8 J, 0. 
Combining this with the left-continuity of £{z\,Z2) in z±, we see that 

l{i«xA5 a } / £{Z{t) - Sue!) du T l {t<aAS }£(Z(t)), 

J 

a.e. (Leb x¥)(t,u) G [0,oo) x O. 

Using the boundedness of £, we now see on taking 5 j in (10) that V~V (x) > 
io 



E J f7A5 e-^£{Z{t)) dt. Since <r € is arbitrary, the result follows. □ 



The next lemma is the second key step in the proof of Theorem 3.1. We 
remark that the special form of the function £ is used crucially in its proof. 



Lemma 3.4. Let x G S. T/ien V + F(x) < u{x). 



Proof. Recall that V{x) = J(x, Y*). Define a* = inf{t > : Y*(t) ■ e\ > 
0}. Note that a* G <S(3>). Let 5 > and define x 5 = x + 5ei. Define the 
stochastic process Z s = {Z^Z^)' as 

Zf(t) = Z^t) + 5, Z 8 (t) = X*(t)=x 2 + B 2 (t)+Y*{t), t>0. 

Let a & = inf{t > : Y*(t) ■ e x > 5}. Note that a 8 > a* and a s j a* as 5 | 0. 
Define 

Y?(t) = (y x *(t) - <5)i {t > CT * } , y 2 5 (t) = y 2 *(t), t > o. 
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Note that X s (t) = x + 5e 1 + B(t) + Y 6 (t) G § for all t > 0. Thus, Y s G A(x + 
8ei,&) and X s is the corresponding controlled process. Also, observe that 
X s (t ) = Z s (t)l {t<a s } +X*(t)l {t > a s } . Next, 

;>oo 

V{x 5 ) < J(x 5 , Y S )=E e-^£(X 5 (t)) dt 
Jo 

= E e-^e(Z s (t))dt + E e-^e(X*(t))dt. 

JO Ja 6 

Thus, 

V(x 5 ) - V(x) < E / e-^\i{Z & {t)) - l(X*(t))) dt 
Jo 

(11) = E / e-^(Z s (t)-X*{t)) 



■ ei Qf £(X*(t) + u{Z 5 {t) - X*(t))) du^j dt, 



where the last line follows from the convexity of I [see (9)]. Recalling that 
l{z\,Z2) is nondecreasing in z\, and that (Z s (t) — X*(t)) ■ e\ > for t < a s , 
we see that 



V(x s ) - V(x) < E / e~~<\Z\i) - X*(t)) ■ e x l(Z\t)) dt 
Jo 

(12) = E r e-T*(Z*(t)-X*(t))-eil(Z*(t))tft 

./o 

+ E / e -^(Z 6 (t) -X*(t)) ■eil(Z s (t))dt. 

J a* 

For t < a*, Y*(t) - ei = and so for such t, (Z 5 (t) - • e\ = 5. Thus, the 

term on the second line of (12) equals <5E/ CT * £(Z 6 '(t)) dt . On the other 
hand, for t G (a*,a s ), Y*(t) -e\ G {0,5) and so for sucht, (Z s (t) - X* (t)) ■ a G 
[0, 5). Thus, it follows that, for arbitrary e > 0, the term on the third line of 
(12) is bounded above by 5ioo(E(a 5 A M(e) - a* A Af(e)) + e), where M(e) 
is such that Jm( £ ) e~ 7 * dt <e and = sup xg g Note that a* < S a.s. 

Using these observations in (12), we obtain 

vc>-n.) sl /-»^ Wl))J 



(13) + i 00 (E[a 5 AM(e)-a* AM(e)]+e) 

< u(x) + F(S) + 4o(E[</ A M(e) - a* A M(e)] + e), 
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where 

(14) F(S) = E / e-^[i{Z\t)) - £(Z(t))} dt. 



o 



Noting that Z s (t) ■ e\ > Z(t) ■ e\ and Z s (t) ■ e 2 = Z(t) ■ e 2 , we have from 
convexity of I that l(Z 5 (t)) > £(Z(t)) and by (7), 

{l(Z 5 (t)) > l(Z(t))} = {Z 5 {t) ■ &l > c~ x Z{t) ■ e 2 > Z{t) • d} 

(15) 

= {0 < V (t) < 5}, 

where r](t) = c~ 1 Z(t) ■ e 2 — Z(t) ■ e\ is a semimartingale with [r/]j = Kt, where 
K = (-l,c- 1 )S(-l,c~ 1 )' G (0,oo). Let (L a ) am be the local time of rj. (We 
refer the reader to Section IV. 7 of [22] for definitions of [-] c and local time.) 
Then for e > and M(e) as before, 

M(e) rM(e) 

^M(v{t))dt = K~ 1 / 1 M (#)) d[r)]$ 



K 1 / _ L M(e) 1 [0,5]( a ) da - 

>0,Ej M ^l m 

and (14) we have 



Thus, for each e > 0, E/ M(e) l m (r](t))dt -» as 5 -» 0. Next, from (15) 



F(5) <e + E ( M{£) e-^(i(Z s {t)) - £(Z(t))) dt 
Jo 

rM{e) 

<e + 2£ 00 Ej^ l m (r](t))dt. 

Combining the above observations, we now have that limsup^o = 0- 
The result now follows on recalling that cr* 5 j a* and taking limits as 5 — > 
and e^O in (13). □ 

Proof of Theorem 3.1. Combining the results of Lemmas 3.2, 3.3 
and 3.4, we have that for each x G § with x ■ e± > 0, 

u(x) < V~V{x) < V + V{x) < u{x). 

Thus, V~V(x) = V + V(x) and, hence, V±V(x) exists for all such x. In a 
symmetric fashion one can show that V 2 V{x) exists for all x G § such that 
x ■ e 2 > 0. The convexity of V yields that ViV is continuous at all x G S 
with x ■ e, > 0, i = 1,2 (see Theorem 25.5 of [23]). Finally, define ViV(x) = 
for x G § with x ■ ej = 0, i = 1,2. To see that this extends continuously the 
definition of W = (Vi V, V2V)' to all of S, it suffices to show that for i = 1,2, 

, _v For each e > 0, there exists <5 G (0, 00) such that 

|ViV(x)| < e whenever < x • e« < <5. 
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We only consider i = 1; the proof for i = 2 is identical. Fix e > and define 
O (e) = £/(24o). Let 5 > be such that 

(17) f( inf (xi + fli(*))>0) <^-, 

whenever < x ■ e\ < 5. Now for each x € § with < x ■ e\ < 5, 

rSAcr 

V 1 V(x)=u(x)= sup E / e-^£(Z(t))dt, 

cre<S(*) Jo 

where <3?, S, Z depend on x and are defined as below Lemma 3.2. Using (17), 
the right side above can be bounded by 



re (e) 

s" 71 dt + ■ 

7 

This proves (16) and the result follows. □ 



I e -it dt + ^p(S>9 (e))<e/2 + e/2 
Jo 7 



4. A related optimal stopping problem. In the remaining sections of the 
paper we will consider the subcase a ^ 0,(3 > 0. Given the convexity and 
nonnegativity of £, this, in particular, implies that a± < 0, oli > 0. We will 
focus first on the case of f3\ > 0; that is, we have 

a x < 0, a 2 > 0, X > 0, 2 > 0. 

The case where (5\ = will be addressed in Remark 5.9. 

We will study an optimal stopping problem and the free boundary as- 
sociated with the control problem in (4). By a suitable re-parametrization, 
we can rewrite the cost in (2) (up to a constant multiplier) as follows: For 

z = (zi,z 2 y € S, 



(18) £(z) 



z 2 - az\ , z 2 > cz\ , 
b ■ z, z 2 < cz±, 



where a € (0, c] and b = (bi,b 2 )' with b\ > and b 2 > 0. From convexity of £, 
it follows that b 2 S [0, 1) and c = jz^- Using the monotonicity of £(z\, z 2 ) in 
the z 2 variable, one can reduce the control problem as follows. For a fixed 
x,$ and Y G A(x,&), let X be as in (3). Define 

(19) Y*(t) = - M {(X2 + B 2 (s)) A 0} 

0<s<t 

and 

(20) X 2 *(t)=x 2 + B 2 (t) + Y 2 *(t), t>0. 

Set X(t) = (X h X^Y, Y = (Fi,F 2 *)'. Clearly, Y G A{x,$) and £{X{t)) < 
£(X(t)). From this it follows that 

(21) U(x)=inf inf Jifx.Fi), 



CHARACTERIZATIONS OF BROWNIAN CONTROL PROBLEMS 



13 



where Ai(x,&) is the class of all RCLL, nonnegative, nondecr easing, {J~t\- 
adapted processes {Y\(t),t > 0} (defined on the system <&) such that x\ + 
Bi(t) + Yi{t) > for all t > 0, a.s. and 



i"0O 

hfaY^ + E e-^H{X{t))dt. 
Jo 



10 

We now introduce the optimal stopping problem associated with this singu- 
lar control problem. For a system and x £ §, let, as before, 5($) be the 
collection of all {^}-stopping times. Also let S,Z,J,£ be as defined below 
Lemma 3.2, but with the new definition of Y£ in (19) and with I defined as 
follows: For z = [zx^z-i)' € S, 



(22) £(z) 



-a, z 2 >czi, 

b\ , Z2 < CZ\ . 



Note that the first coordinate of Z is a Brownian motion, while the second 
coordinate is a reflected Brownian motion; in particular, Z has continuous 
sample paths. Consider the optimal stopping problem of choosing a stopping 
time a to maximize the reward in (6) with S, Z, I as described above. Then 
the value function for the optimal stopping problem for initial condition x 
is defined as 

(23) u{x) = sup sup J{x,a). 

* o-e5(<i>) 

Note that clearly u(x) < co and taking a = 0, we have u(x) > for all x € S. 

The proof of the following theorem is analogous to that of Theorem 3.1, 
so we only provide a sketch. 



Theorem 4.1. For every x € S, ViV(x) = u(x). 



Sketch of proof. Let x € S with x ■ e.% > 0. Let $ be an arbitrary 
system and let a € 5( < ^ > )- Fix e > 0. From Proposition 2.1 we can find a 
Y\ 6 A(x,$>) such that Ji(x,Y\) < V(x) + e. In fact, Proposition 2.1 says 
that Y± can be chosen to be adapted to {J-®}- Following the proof of Lemma 
3.3, we see that 

(24) V(x s )- V(x) >_g +E r A54 e -7t f 1 i(z(t)-5u ei )dudt, 
—o Jo Jo 

where Sg is as in the quoted lemma. Note that the second expression on 
the right side above does not depend on e (or on the e-optimal control 
Y\). Letting e — > and then 5 j 0, we now have, on recalling that a and $ 
are arbitrary, that ViV(a?) > u(x) for all x 6 § with x ■ e\ > 0. The proof 
of ViV(x) < n(x) for all x G S is identical to that of Lemma 3.4. Hence, 
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WiV(x) = u(x) for all x £ S with x ■ e\ > 0. Finally, when x ■ e\ = 0, both 
ViV(a?) and it (a:) are zero. This proves the result. □ 

The following corollary gives a useful characterization of an optimal stop- 
ping time in terms of an optimal control. 

COROLLARY 4.2. Let x e § and suppose $ and F* 6 A^x,*!?) are such 
that V(x) = Ji(x,Yi) (existence of such a Y* is guaranteed from Proposition 
24). Let 

(25) a* =M{t >0:Y{(t) > 0}. 
Then u{x) = J(x, a*). 

Proof. Taking limits as 6 — > and e — > in the first line of (13), we 
have that 

u(x) = ViV(x) = V + V(x) < E / e-^l(Z(t)) dt < u(x). 

Jo 

The result follows. □ 

The above corollary establishes the existence of an optimal stopping rule 
given on some filtered probability space. The following proposition shows 
that, in fact, the optimal rule can be chosen to be an { Tf }-stopping time 
and so in the optimal stopping problem (described above Theorem 4.1) it 
suffices to optimize over a fixed system $ with the filtration taken to be 
{^ B }. Note that from the C 1 property of V established in Theorem 3.1 we 
have that u is continuous on S. 

Proposition 4.3. Let x e S and & be a system. Define 

(26) a = 5Ainf{t>0:n(Z(t))=0}. 
Then a® is optimal; that is, u{x) = J{x,uq). 

The key step in the proof of the proposition is the following lemma. 

Lemma 4.4. Let x e S and let $ and Y{ e Ai(x±,&) be as in Corollary 
4-2. Define, for fixed e > 0, 

(27) a £ = S Ami {t>0:u(Z(t)) < e}. 
Then a e < a* a.s., where a* is as in Corollary 4-2. 
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PROOF. Define 9 £ = a £ Aa* and 9 £ = 9 £ + (5 A (a £ - (x*) + )l{ (7 » <00 }, where 
5 > is chosen so that ee~ lS — £n p 5 > e/2, and £u p > is the Lipschitz 
constant for the function t. Let Y\{t) = — min(0, m£g s < s <t \X\ (0 £ — ) + Bi(s) — 
Bi{9 £ )}), t>9 £ , and define for t > 0, 

(28) yiW^^Wl^^WlK^I + ^W 1 ^,-)^)!^^}- 

Note that Xx(t) = x± + -Bi(i) + > for all t > and, consequently, 
Yi G ^4i(xi,$). In particular, 



V(x) <E 



where X = (XijXj)'. Also, from the optimality of Y*, we have that 



V(x)=E 



-~<H{x*{t)) dt + e -^y(x*(e £ )) 



Observing that Yi(t) = Y%(t) = for t < 9 £ , we have, on combining the above 
two displays, 

E{e-^(V(X*(9 £ ))-V(X(9 £ )))]<E f' e-*[l{X(t)) - l{X*(t))\ dt. 



On the set {a £ <a*}, 9 £ = 9 £ and Yi(t) = Y{(t) for all t > 0, and thus, on 
this set, the expressions on both the left and right of the above inequality 
are 0. Thus, we have 



E(i {as>a * } e-^(v(x*(e £ ))-v(x(e e )))) 



(29) 



<E 1 



l {<Te><T*} 



^[£(X(t))-£(X*(t))]dt). 



Using the convexity of V, Theorem 4.1 and the definition of a e , we have on 
the set {a £ > a* } 

V(X*(9 £ ))-V(X(9 £ )) 

= {xi(9 £ ) -Me,)) ^^{Xxie^+vixi^-Me^xiie^dv 

Jo 

> (XZ(S e ) -Xi(0 e ))u(Z(0 e )) > eY{(9 £ ). 

Thus, 



(30) E[ £ l K> ^ } e-^yr(^)]<E[l { , £>(T . } e-^(y(X*(^))-y(X^)))]. 
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Next, using the Lipschitz continuity of t and that Y*(t) > Yi(t),t > 0, we 
have 



E 



l{cr £ >o-*} 



-^[e(x(t))-e{x*(t))]dt 



(31) 



<E 



l{ CTE > CT *}e' 



< E[£ ]ip i {ae>a . } e-T e <Y;(e e )(e e - e e )\. 



Combining (29), (30) and (31), and recalling that on the set {o~ £ > <r*}, 
9 £ = a* and 9 £ = a* + 5 A (a £ - a*), we have 

(32) E[l WE>a , } e~^Y{(6 £ )(ee-^ A ^-^ - £ Up (6 A (a £ - a*)))} < 0. 

By the choice of 5, the term ee~^ 5 ^ ae ^ (T *^ - £\ ip {5 A (a £ - a*)) is between e 
and e/2. Thus, (32) implies 



(33) 



E[l K>CT , } e-^*>m)]=0. 



Finally, on the set {a £ > a*}, since 6 £ > a* , recalling the definition of a*, we 
have Yi*(0 e ) > 0. Equation (33) then implies that F[a £ > a*} =0. □ 

Corollary 4.5. For allx € § and any arbitrary system u £ as defined 
in (27) is an e-optimal stopping time, and we have 



(34) 



u(x) = sup J(x, a), 
<reS B 



where Sb is the set of all {J 7 ®} -stopping times. 

Proof. Note that the right-hand side of (34) is independent of the 
choice of the system so without loss of generality we can take <3? to be 
the system on which a* [as defined in (25)] is given. Let e > be arbitrary. 
From Lemma 4.4 a £ < a* a.s. and, thus, 



u{x) =E 
(35) = E 



CT*AS 



-^l{Z{t))dt 



r*A5 



-7(t-<T e 



] £{Z{t))dt 



< J(x, a £ ) + E[e-^u(Z(a £ ))} < J(x, a £ ) + e, 



where the last line follows from the definition of a £ . Thus, a £ £ 5b is an 
e-optimal stopping time. Since e > is arbitrary, the result follows. □ 



CHARACTERIZATIONS OF BROWNIAN CONTROL PROBLEMS 



17 



Proof of Proposition 4.3. Let a e be as in (27). Since a £ is nonde- 
creasing in e, a + = lim e _>oCT' E exists a.s, and clearly, a + £ Sb with a + < S 
a.s. Note that a + is an optimal stopping time, since, as e — > 0, 

/■SAo> rSAcr+ 

J(x,a £ )=E e-^i{Z(t))dt^¥, e - ^(Z(t)) dt = J(x,a + ). 

Jo Jo 

We now show that a + = <jq , thus completing the proof. Clearly, a + < oo . 
From continuity of u and Z we get that u{Z{a e )) — > u(Z(o~ + )), a.s. on the 
set {o" + < 5}. Recalling the definition of a E , we now see that on the above 
set u(Z(a + )) must be 0. The inequality o~o < cr + follows. □ 

Next, in preparation for the free boundary characterization and the con- 
jectured form for the optimal control for the control problem in (21), we 
summarize below some key properties of u. 

Lemma 4.6. The function u:S — > [0, oo) satisfies the following proper- 
ties: 

1. For x = (xijXz)' G S, if X2 < cx±, then u(x) > 0. 

2. For x = (x±,X2)' € S, u(xi,X2) is nondecreasing in x\ and nonincreasing 
in xi- 

3. u is continuous on S. 

4. For all x\ > 0, there is X2 < oo such that u{x\,X2) = 0. 

Proof. Let x G S be such that X2 < cx\. Let $ be an arbitrary system 
and let t\ = inf{i > : Z^ify > cZ\(t)}, where Z is as introduced above (22). 
Since t\ > and S > a.s., we have 

u(x) > J(x,n) = (6i/7)E[l - e~ 7 ( TlA5 )] > 0. 

This proves part 1. Monotonicity of u(xx, X2) in x\ follows from the convexity 
of V, while the monotonicity in X2 is an immediate consequence of the fact 
that £(x\,X2) is a nonincreasing function of X2- Continuity of u, as noted 
earlier, is a consequence of Theorems 4.1 and 3.1. 

We now consider part 4. Fix < e < 1/2 small enough so that jr^(o + 
b\) < a. Let S > be such that j^(a + b\) < 5j < a. Choose T G (0, 00) to 
satisfy 

pT poo 

-a / e" 7S ds + bi / e" 7S ds = -5, 
Jo Jt 

that is, T = — i log ~^?^ a . We will argue via contradiction. 

Suppose there exists < x\ < 00 such that u(xi,X2) > for all X2 > 0. 
Let $ be an arbitrary system and let x\ > x\ be such that F{S < T] < e/2, 
where S = inf {t > 0:x x + B x (t) = xi}. Fix x 2 > cx\ so that P[r < T] < e/2, 
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where r = inf{t > : Zz(t) < cZi(t)}, with x = (xi,X2)' ■ Then by part 2 and 
our assumption, we have u{x) > 0. Note that £(Z(t)) = —a for all t < r. By 
Proposition 4.3, u(x) = J(x,o~o), where do is defined in (26). We next claim 
that S < o"o a.s. To see the claim, note that for all t > 0, 

CT ° At e-^ s i{Z{s)) ds + e-^ oAt K{Z{a A t)) 



u(x) = E 

> J(x, a At) + K[l {§>a()} e^°u(Z(a A *))]. 
Letting t —> oo, we see that 

u(x) > u(x)+E[l {§>r7o} e-^u(Z(a ))]. 

Thus, E[l { g >(7o} e-^ <T0 u(Z(o- ))] = 0. Since, on the set {S > cr }, e~ 7<T ° u(Z (<t )) 

is strictly positive, it follows that P(5 > cro) = 0. This proves the claim. Note 
that almost surely on the set {t >T,S >T}, 

/ e-^i(Z(t))dt<-a e'^dt + h 
Jo Jo Jt 



e^dt 
it 



Writing 

1 = {r>T,S>T} + ^{t<T} + 1{5<T} ~~ ^{t<T,S<T}i 

we have 



E 



H e-^£(Z(t)) dt < -SP[t >T,S>T} + (6i/ 7 )(P[t <T}+ F[S < T]) 
Jo 



+ (a/ 7 )P[r < T, 5 < T] 

<-<5(l-e)+e(a + 6i)/ 7 . 

The quantity in the last line above is less than from our choice of 6. Thus, 
we have shown that u(x) = J(x, a"o) < 0, which is a contradiction. Part 4 
now follows. □ 



5. The free boundary and an optimal control policy. Recall that we re- 
strict ourself to the case ot\ < 0, a 2 > 0, (3\ > 0, /3 2 > 0, which after suitable 
re-parameterization leads to the running cost (18). Guided by Lemma 4.6, 
we now introduce the free boundary, d* = {(^(z^), .z 2 ), Z2 > 0}, where ^ is a 
map from M + to R+, associated with the optimal stopping problem in (23). 
Let 

(36) tf(* 2 ) =sup{zi >0:n(zi,z 2 ) =0}, z 2 €[0,oo). 

The following lemma summarizes some key properties of 

Lemma 5.1. The function has the following properties: 
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1. 0<$(2 2 )<VC,Z2>0. 

2. is nondecreasing. 

3. $ is Lipschitz continuous on R + : For all Zi,zi G K+, 

|*(z 2 ) - #(32) | < c _1 |z2 - z 2 \. 

4. lim Z2 ^ 00 ^(z 2 ) =00. 

PROOF. Clearly, u(0,X2) = for all X2 > 0, which implies that the set 
in (36) is nonempty. This along with part 1 of Lemma 4.6 shows that is 
a well defined map from M + to K + and it satisfies the inequality in part 1 
of the current lemma. Part 2 is a consequence of part 2 of Lemma 4.6. Part 

4 follows from part 4 of Lemma 4.6. It remains to prove part 3. 

Since \& is nondecreasing, it suffices to show that for every Z2 > and 
h>0, 

(37) ^(z 2 + h)<^(z 2 )+c~ 1 h. 

From the definition of \& we see that to prove (37) it is enough to show that 

(38) u(x\ + c _1 /i, X2 + h) > u(x\, X2) for all (x±, x 2 ) G R?_. 

Let a be an optimal stopping time for the initial condition x = (xi,X2), that 
is, 

rcrAS 

u(x)=E / e-^£(Z(t))dt, 
Jo 

where Z is as defined in (5) with Y 2 * given by (19). Note that 

paAS rcrAS 

u(x) = (61 + a)E / e" 7 * l Lc (Z(t)) dt-aE e" 7 * dt, 
Jo Jo 

where L c = {(21, 2:2) G : Z\ > c -1 ^}. Let Z be defined by the expression 
in (5) with x there replaced by x = (xi + c~ 1 h, X2 + /i) and replaced 
by Y 2 *, which is defined by (19) with X2 replaced by X2 + h. Note that 

5 = inf {t > : Z\ (t) = 0} > 5 a.s. Thus, we have 

rcrAS 

u{x) > J(x, a A 5) = E / e~^i{Z(t)) dt 
Jo 

rcrAS rcrAS 

= (b 1 +a)E e -^l Lc (Z(t)) dt-aE e'^ dt. 
Jo Jo 

Thus, in order to prove (38), it suffices to show that 

(39) Z(t) G L c Z(t) G L c for all t, a.s. 
Finally, note that 

Z(t)eL c Z 1 ( y t)>c' 1 Z 2 (t) +c' 1 h> C - 1 (Z 2 (t) + h) 

z Y (t) > c" 1 z 2 (t) =>■ z(t)eL c , 
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where we have used the fact that Z\(t) = Z\(t) + c~ 1 h and Z^t) < Z^ii) + h 
for all t > 0. This proves (39) and the result follows. □ 

Proposition 4.3 and Lemmas 4.6 and 5.1 lead to the following candidate 
for an optimal control policy. 

Fix x G § and let be an arbitrary system. Let 

(40) Y{(t) = -minjo inf [x, + B^s) - *(X 2 *(a))]}, 

L o<s<t j 

where X| is as in (20). The remaining section is devoted to the proof of the 
following theorem. 

Theorem 5.2. For all ieS, 

J 1 (x,Y?) = J(x,Y*) = V(x), 
where Y* = (Y*,^)' with Y 2 * as defined in (19) and Y* as defined in (40). 

We begin with the following lemma. 

Lemma 5.3. For all x G X with x ■ e 2 > 0, V 2 y(z) > 0. 

Proof. Recalling that V is convex, it suffices to show that V(x h ) > 
V(x), where x = (xi,X2)' G and x h = x + /ie 2 . Let Y\ G Ai(x h ,$) for 
some system $ be such that 

(41) V(x h ) = J 1 (x\Y 1 ). 

Note that since x ■ e\ = x ■ ex, Y± G Ai(x, <&) as well. Thus, we have that 



CO 



V(x)<E e-^£(X{t))dt, 



where X(t) is as defined below (20) with Y\ as chosen above. Let X h = 
(A > i,X 2 ' /l ), where X^ is defined via (19) and (20) with x 2 replaced by x 2 + 
h. Note that V(x h ) = E/ °° e~"iH{X h {t)) dt. Let rj h = m£{t >0:X%' h (t) = 
0}. Since x 2 + h > 0, we have that % > a.s. Also note that I is strictly 
increasing in the second variable, that is, l(z + re 2 ) > l{z) for all z G Ml 

and r > 0. Combining these observations with the fact that X 2 '' l (t) > X%(t) 
for all t G [0,%), we see that, a.s., 

Vh e-~ ft l(X(t)) dt < e-^£{X h {t)) dt. 



Also, 

roo roc 



e T t £(X(t))dt= / e^H{X h {t))dt. 

Vh J Vh 
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Combining the above with (41), we have V(x) < V(x h ). This proves the 
result. □ 

The following result was established in [1]. Let A be the second order 
operator 

A= -±tr(£L> 2 ) -6-D, 
where D denotes the gradient vector and D 2 the Hessian matrix. 

Theorem 5.4 (cf. Theorem 2.1 [1]). V is a constrained viscosity solution 
of the PDE 

(42) (rfip(x) + Aip(x) -£{x)) V m&x(-Di)(x) ■ e*) = 0, x € §. 

i=l,2 

Namely, 

(i) V is a su^er solution of (42) on §: For all x € § and all cp € C 2 (§) 
for which V — (ft has a global minimum at x, one has 

(jV(x) + A<t>{x) - £{x)) V rnax(-D0(s) • a) > 0. 

(ii) V is a subsolution of (42) on S°: For all x G §° and aZZ G C 2 (S) 
for which V — <j) has a global maximum at x, one has 

(jV(x) + A4>(x) - l(x)) V msx{-D<j>{x) ■ t%) < 0. 

i=l,2 

Let G = {x = (xi,X2) € -X\ > ^(x2)}. Note that the interior of G is 
given as G° = {x £ G : x\ > ^(x2),X2 > 0} and the boundary of G, dG, sat- 
isfies dG = diG U d 2 G, where d±G = {x e G : x± = V(x 2 )} and 9 2 G = {x e 
G :X2 = 0}. For B C S, we denote V restricted to D as V\o- 

Lemma 5.5. V\ G o e C 2 (G°) and 

(43) jV(x) +AV(x) -l{x) =0 forallxeG . 

Proof. From Theorem 5.4, V is a viscosity solution of (42) on G°. From 
Lemma 5.3 and the definition of we see that for all x 6 G°, ViV(x) > 0, 
i = 1,2. This shows that V\g° is a viscosity solution of (43) on G°. Indeed, 
let x G G° and 4> G C 2 (G°) be such that x is a minimum point of V — In 
particular, Vi<j)(x) = ViV(x) > for i = 1,2. From Theorem 5.4(i) we must 
then have that 'yV(x) + A(p(x) — £(x) > 0, showing that V\g° is a viscosity 
supersolution of (43) on G° . The subsolution property is immediate. By 
standard elliptic regularity results (see, e.g., Theorem 6.13 of [10]), one then 
has that V\qo £ C 2 (G°) and it is a classical solution of (43) on G°. □ 
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Define for e > 

G £ = {xGM 2 :d(x,G) <e}, 

where for x G M 2 , d(x, G) = infygc \x — y\. For x G §, we define x(e) = x + 
e(e), where e(e) = e((2 + 3c" 1 ), 2)'. 

Lemma 5.6. x(e) G G° /or ewery x G G £ . 

Proof. Fix x G G £ . Then for some x* G G, 

(44) |x • ei — x* ■ ei| < e, |x • e2 — x* ■ e 2 | < e. 
Using (44) and the fact that x* £ G, we have 

x(e) • e2 = x • e2 + 2e > x* • e2 + e > e. 

Also, 

x(e) • e\ = x ■ e\ + (2 + Sc" 1 )^ 

> x* • ei + (1 + 3c _1 )e > ^(x* • e 2 ) + (1 + 3c _1 )e 

> *(x(e) • e 2 ) - c _1 |x* • e 2 - a;(e) • e 2 | + (1 + 3c _1 )e 
>^(x(e)-e 2 )+e, 

where the first inequality in the above display follows from (44); the second 
follows since x* G G, the third inequality uses Lemma 5.1 (3) and the last 
inequality is a consequence of \x* ■ e2 — x(e) ■ e 2 | < 3e. The result x(e) € G° 
follows on combining the above two displays. □ 

Define V s on G £ by the relation 

V £ (x) =V(x(e)), xgG £ . 
Note that by Lemmas 5.5 and 5.6 y £ is G 2 on G £ , and 

(45) DV e (x) = DV(x(e)), D 2 V £ (x) = D 2 V{x(e)), xGG £ . 



Lemma 5.7. There exists p G (0, oo) smc/i t/iat /or all e G (0, 1) and x G 

G £ 

|7^ £ (x) +AV £ {x) -i{x)\ <pe. 

Proof. From Lemma 5.5 and (45), we have that for all x G G £ 

jV £ {x) + AV £ (x) - £{x) = 7V r (x(e)) + AV(x(e)) - £(x(e)) + £(x(e)) - 

= £(x(e))-£(x). 
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The result now follows on using the Lipschitz property of I and the definition 
ofx(e). □ 

The following moment estimate is an immediate consequence of the Lip- 
schitz property of \& and standard moment estimates for the running maxi- 
mum of a Brownian motion. The proof is omitted. 

Lemma 5.8. For every p > 1, there exists c p 6 (0, oo) such that for each 
x £ § and all t>0, 

maxE|y;*(t)| p + E sup \X*(s)\ p < c p (l + \x\ p + t p / 2 ). 

» =1 > 2 0<s<t 

We now proceed to the proof of Theorem 5.2. 

Proof of Theorem 5.2. We first consider the case where i£G. We 
will apply Ito's formula to the semimartingale X* and the function (t, x) i— > 
e~ lt V £ {x). Note that X* takes values in G and V £ is C 2 on G £ , an open set 
containing G. Thus, 

Ee-^V £ (X*(t)) = V e {x) - E f e^ s (jV £ (X*(s)) + AV £ (X*(s))) ds 

Jo 

+ f> fv t V £ {X*{s))dY*{s). 

Here we have used the fact that, since V is (globally) Lipschitz on G, DV e 
is uniformly bounded on G £ . Using Lemma 5.7, we now have 

V £ {x) + 7 ~ V > ^e-^V £ (X*(t)) + E f e-^ s e{X*{s)) ds 
(46) _ t h 

~~7 JO 



2 

'E 
i=i Jo 

Next, using the C 1 property of V, we have that, for x € G, i = 1, 2, 

\ViV £ (x) - ViV{x)\ = \ViV(x + e(e)) - ViV(o;)| -► 0, as e -c 0. 

Using this along with Lemma 5.8 and noting that sup xeG sup eg ( ,i) I VjU e (x)| < 
oo, we get, for i = 1, 2, 

E f t V t V £ (X*(s))dY*(s)^E f'viViX^s^dY^s) 
Jo Jo 

as e — > 0. Another application of Lemma 5.8 and recalling that V has at 
most linear growth (Lemma 2.2) gives that 

Ee-^V £ (X*(t)) ^Ee- 7t U(X*(t)) 
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as e — > 0. Taking limits as e — > in (46), we obtain 

V(x) > Ee-^V(X*{t)) + E f e-^ s £{X*(s)) ds 

Jo 

-^E/'viF(x*( s ))dy;( s ) 

= Ee-^V(X*{t)) + E / e -^(X*(s)) ds, 
Jo 

where the last equality follows on noting that Y*(s) increases only when 
X*(s) G diG and ViV{x) = for x G <9jG (see the proof of Theorem 3.1 and 
the definition of ^.) Next, using Lemma 5.8 once more and recalling the 
linear growth property of V, we see that Ee _7 *V (X* (t)) — > as t — > oo. The 
assertion in the theorem, for i£G, now follows on taking t — > oo in the 
above display. 

Now consider x = (xi,x 2 ) G B\G. In this case, Yi*(0) = *(x 2 )-xi, X*(0) = 
{^{x 2 ),x 2 ) G G, and thus, 

Jr(xi,x 2 ) = Jf(V(x 2 ),x 2 ) = V(V(x 2 ),x 2 ). 

Also, by Theorem 4.1, ViV(x) = u(x), which is for x G § \ G. Thus, 
V(^(x 2 ),x 2 ) = V(xi,x 2 ) and, hence, J*(x) = V(x) for all x G S. □ 

Remark 5.9. We now consider (3\ = 0. In this case, &i = and, thus, 
£(z) = —al Z2 > CZl < 0. It follows that u{x) = for all x G S and so '3/ cannot 
be defined via (36). Instead, we define 

(47) Y?(t) = - minjo, jnf >x + B 1 (s) - X 2 »/c]}, 

where X 2 is as in (20). Let x G S satisfy x 2 < cx\. Then X|(t) < cXJ (i) and, 
thus, £(JT*(t)) = 62^2 (*) for all t > 0. Noting that £(z u z 2 ) > b 2 z 2 , (zi,z 2 ) G 
§, we then have that V(x) = J(x,Y*). Proof that V(x) = J(x,Y*) for x 
with x 2 > cxi follows as in the last paragraph of the proof of Theorem 5.2 
on noting that ViV(x) = u(x) = 0. 
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